Video-Based On-line Handwriting Recognition

The use of handwriting provides a natural way of interacting with small portable computers. However, in order to capture handwritten text on-line special input devices are necessary. Therefore, Munich & Perona [13] proposed to use visual input for pen-based computers. Writing can then be performed on ordinary paper and pen trajectories are automatically extracted from image sequences recorded during the writing process. On the basis of this work we developed a complete video-based on-line handwriting recognition system. We will present the techniques applied for pen tracking, pre-processing, feature extraction, and statistical modeling and recognition. Evaluation results on a writer independent unconstrained handwriting recognition task demonstrate that the inherent limitations of the videobased approach can be compensated using robust modeling combined with adaptation techniques.